Extending Spectral Methods to New Latent Variable Models

Latent variable models are widely used in industry and research, though the problem of estimating their parameters has remained challenging; standard techniques (e.g., Expectation-Maximization) offer weak guarantees of optimality. There is a growing body of work reducing latent variable estimation problems to a certain(orthogonal) spectral decompositions of symmetric tensors derived from the moments of observed variables. Such decomposition allows a robust and computationally tractable estimation approach for several popular latent variable models; examples include topic mixture models, mixture of gaussians and HMM. In this report, we extend spectral methods to yet another class of latent variable models—topic transition models and mixture language models.